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Abstract 

Game dynamics in which three or more strategies are cychcally competitive, as repre- 
sented by the rock-scissors-paper game, have attracted practical and theoretical interests. 
In evolutionary dynamics, cyclic competition results in oscillatory dynamics of densities 
of individual strategists. In finite-size populations, it is known that oscillations blow up 
until all but one strategies are eradicated if without mutation. In the present paper, 
we formalize replicator dynamics with players that have different adaptation rates. We 
show analytically and numerically that the heterogeneous adaptation rate suppresses the 
oscillation amplitude. In social dilemma games with cyclically competing strategies and 
homogeneous adaptation rates, altruistic strategies are often relatively weak and cannot 
survive in finite-size populations. In such situations, heterogeneous adaptation rates save 
coexistence of different strategies and hence promote altruism. When one strategy domi- 
nates the others without cyclic competition, fast adaptors earn more than slow adaptors. 
When not, mixture of fast and slow adaptors stabilizes population dynamics, and slow 
adaptation does not imply inefficiency for a player. 

Keywords: cyclic competition, replicator dynamics, altruism, coexistence 
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1 Introduction 



Cyclic competition among different phenotypes is often found in nature. A minimal system of 
cyclic competition consists of individuals of three phenotypes. In so-called rock- scissors-paper 
(RSP) dynamics, R beats S, S beats P, and P beats R, hence no phenotype is entirely dominant. 
Examples of natural RSP dynamics include tropical marine ecosystems (Buss, 1980), color 
polymorphism of natural lizards (Sinervo and Lively, 1996), vertebrate ecosystems in high-arctic 
areas (Gilg et al., 2003), real microbial communities of Escherichia coli (Kerr et al., 2002), and 
various disease dynamics represented by the susceptible-infected-recovered-susceptible model 
or its variants (Anderson and May, 1991). 

Also in evolutionary game theory, cyclic competition dynamics realized by three or more 
strategies often appear. In social dilemma games such as the Prisoner's Dilemma, cyclic com- 
petition often underlies the survival of altruistic strategics subject to invasion by nonaltruistic 
strategies. For example, if the players can take either unconditional cooperation (ALLC) or un- 
conditional defection (ALLD) in the Prisoner's Dilemma, ALLD eventually prevails. However, 
if discriminators that use reputation of other players regarding the cooperation tendency are 
allowed, ALLC, ALLD, and discriminators form an RSP cycle; ALLD is stronger than ALLC, 
ALLC and discriminators are neutral, and discriminators are stronger than ALLD under certain 
conditions. Accordingly, these strategies alternately prosper in the population, which implies 
survival of altruistic players, namely, ALLC and discriminators (Nowak and Sigmund, 1998a; 
Nowak and Sigmund, 1998b). Alternatively, if players can choose not to play the pubhc goods 
games, such players called loners beat ALLD, ALLD beats ALLC, and ALLC beats loners. 
The elicited RSP dynamics save altruistic ALLC players, which would not survive without 
loners (Hauert et al., 2002a; Hauert et al., 2002b; Semmann et al., 2003). In the iterated 
Prisoner's Dilemma, a mixture of ALLC, ALLD, and tit-for-tat (TFT) constitutes RSP dy- 
namics so that altruism survives under small mutation rates (Imhof et al., 2005). Finally, a 
population of different types of strategies, each of which is based only on the last actions of 
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players, shows periodic and chaotic oscillations of population densities associated with cyclic 
competition (Nowak and Sigmund, 1989a; Nowak and Sigmund, 1989b; Nowak and Sigmund, 
1993; Brandt and Sigmund, 2006). 

As illustrated by these examples, an important consequence of cyclically competing evolu- 
tionary dynamics is oscillations of population densities. Each strategy or phenotype alternately 
becomes the majority until it is devoured by its enemy. Oscillations are usually self- destructive 
in small populations because the finite-size effect usually drives the evolutionary dynamics to 
eventual dominance of one of the competing phenotypes (Taylor and Jonker, 1978; Nowak and 
Sigmund, 1998b; Reichenbach et al., 2006). If the model represents a ecosystem, dominance of 
one species implies a loss of biodiversity. In social dilemma games, a situation in which one 
strategy wins cyclic competition often implies a loss of altruism; in many models, an altruistic 
strategy is relatively weak compared to others and beats another strategy only barely (Nowak 
and Sigmund, 1998b; Hauert et al., 2002b; Imhof et al, 2005). 

If oscillations are stabihzed or suppressed, cyclically competing strategies will coexist in 
a population. Oscillations can be suppressed by introduction of spatial structure such as the 
square lattice (Hassell et al., 1991; Tainaka, 1993; Durrett and Levin, 1998; Frean and Abra- 
ham, 2001). In line with this, oscillation amplitudes grow as the underlying population struc- 
ture transits from the regular lattice to the well-mixed population by increasing the number of 
long-range edges (Szabo et al., 2004; Ying et al., 2007) (see (Szabo and Fath, 2007) for more 
references). Alternatively, heterogeneous contact rates of individuals suppress oscillations. In 
this scenario, hubs, namely, individuals with many neighbors, are more susceptible to invasion 
by their enemies, because hubs have many enemies in the neighborhood than non-hubs on aver- 
age. Then hubs complete a state-transition cycle in shorter periods. Mixture of individuals with 
different oscillation periods abohshes the oscillation of the entire population density (Masuda 
and Konno, 2006). 

An implicit assumption made by these and many other studies of cyclic competition (Szabo 
and Fath, 2007) is that a focal individual is invaded by each enemy neighbor independently at 
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a constant rate. For example, if a focal individual is in state R and it has 10 neighbors, the 
state of the focal individual is replaced by state P, which governs R, at a rate proportional to 
the number of P among the 10 neighbors. In other words, each P individual preys on each R 
neighbor independently at a constant rate. The contributions of different P individuals add 
linearly. In evolutionary games, however, the effect of each neighbor adds linearly in the sense 
that the aggregated payoff of the focal player over is the linear sum of the payoff obtained 
by playing with each neighbor. Then natural selection operates on the aggregated payoff in 
a nonlinear manner. Here how one player elicits state transition of another player is not 
straightforward . 

In addition, suppression of oscillations by the heterogeneity mechanism requires that the 
number of contacts, or neighbors, depends on individuals (Masuda and Konno, 2006). In the 
context of evolutionary games, heterogeneous contact numbers also pave the way to increased 
altruism (Santos and Pacheco, 2005; Santos et al., 2006). In this situation, enhanced coop- 
eration stems from players with many contacts, who earn more by participating in the game 
more often. However, how to compare aggregated payoffs of players with different numbers of 
participation is an unresolved issue (Masuda, 2007). 

In this work, we explore how oscillations in RSP dynamics can be stabilized or suppressed in 
evolutionary games. To avoid the problem of the heterogeneous number of contact per player, 
we consider a well-mixed population and assume that the adaptation rate, but not the con- 
tact number, depends on players. Some players are quicker to copy successful strategies than 
others. Such heterogeneous adaptation rates are known to enhance cooperation in the evolu- 
tionary Prisoner's Dilemma on the regular lattices and small- world networks (Kim et al., 2002; 
Szolnoki and Szabo, 2007). By extending the replicator dynamics, we show that heterogeneous 
adaptation rates suppress oscillations of population densities in well-mixed populations. We 
validate our theory by numerical simulations of the RSP games and the public goods game 
with loners (Hauert et al., 2002a; Hauert et al., 2002b). 
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2 Model 

2.1 Replicator dynamics 

We begin with the standard rephcator dynamics. Suppose that there are m possible phenotypes, 
which we call strategies. The proportion of individuals, which we call players, with strategy i 
in the population of n players is denoted by Xj (1 < i < m), where J^i^i = ^ and < Xj < 1 
(1 < z < m) are always satisfied. The fitness of strategy i is denoted by /j. The standard 
replicator dynamics are represented by 



dt 



■^•i I fi ^ ^ fj'^j 



= XiY,{fi- fj)Xj, (1) 
j 

where we suppress the range of j when no confusion arises. Players with strategy i increase 
in number at a rate proportional to the excess of fi relative to the average fitness J2jfjXj- 
Put another way, a player with strategy j switches its strategy to strategy i of a randomly 
picked opponent at a rate proportional to the fitness difference fi — fj, if fi — fj > 0. If 
fi — fj < 0, players with strategy i may switch to strategy j. The players are interpreted to 
form a well-mixed population. 
We rewrite Eq. (pO) as 

^ = XiY^ Gifi - fj)xj -XiYl G(fj ~ fi)^v (2) 
j j 

where 

The first term of Eq. ([2]) represents the inflow of the population density to strategy i and the 
second term represents the outflow from i to other strategies. Note that 

G{x) - G{-x) = X (4) 

is satisfied. 



2.2 Extended replicator dynamics 

We extend the standard replicator dynamics by considering a heterogeneous population of 
players who differ in the adaptation rate. For simplicity, we assume only two adaptation rates 

and f3y. The corresponding proportions of each strategy in two different subpopulations are 
specified by Xi and Hi {I < i < m), where J^i^i = J^iVi = and < Xj, < 1 (1 < i < m). 
In a homogeneous population, the adaptation rate, which is common to all the players, can 
be ignored by rescaling the time, as is evident in Eq. ([1]). With two adaptation rates, the 
ratio Px/Py is an essential parameter that may alter dynamical consequences. Note that the 
interaction rate is common to all the players, which contrasts with Taylor and Nowak (2006). 

If one strategy is more profitable than another strategy irrespective of the values of Xi 
and i/i {1 < i < m), players that adapt more rapidly enjoy more benefits. For example, 
in the Prisoner's Dilemma with ALLC and ALLD strategies only, everybody in a well-mixed 
population will choose ALLD. Rapidly adapting ALLC players switch to ALLD in early stages 
to transiently exploit slowly adapting ALLC players. However, under cyclic competition, it 
may not be profitable to myopically follow apparently successful others. In a long term, players 
may be better off by adapting slowly. 

We assume that players differ only in adaptation rates and that they are well-mixed. There- 
fore, the total payoff of a player with strategy i is equal to fi regardless of the adaptation rate. 
We represent the extended replicator dynamics as follows: 



1 dxi 



fi-J2 fj^j ] +yiJ2 '^ifi - fi)^j G{fj - fi)yj (5) 



= yiyfi-J2 fiVjj G{fi - fj)yj -yiY. G{fj - fi)xj. (6) 

In Eq. (ini), the first term in the right-hand side represents the competition within players that 
have adaptation rate jSx- This part is identical to the standard replicator dynamics. The second 
term represents the transition from strategy j to strategy i as a result of imitating a player 
with the different adaptation rate, that is, jSy. Players with strategy j and adaptation rate jSx, 

7 



whose density is Xj, switch to strategy i when they meet a player with strategy i and adaptation 
rate f3y. The transition rate is proportional to G{fi — fj). Note that the transition from j to 
i such that players with strategy i and adaptation rate jSx are imitated is taken care of by the 
first term. The third term represents the rate at which players with strategy i and adaptation 
rate (3x switch to strategy j when they meet a player with strategy j and adaptation rate jSy. 
Equation ([6]) can be interpreted similarly. Equations (I5l) and ([6]) guarantee J2idxi/dt = and 
J2i dyi/dt = 0, so that the number of players in each subpopulation is conserved. 

Cyclic competition often yields coexistence of all the relevant strategies. Interested in this 
phenomenon, we focus on interior equilibria, which satisfy Xj > (1 < Vi < m). In the standard 
replicator dynamics shown in Eq. ([T]), suppose that the total payoff for each strategy fi is linear 
in xi, X2, . . ., Xm, as is the case when the payoff is defined by a payoff matrix. Then the interior 
equilibrium is unique and determined from /i = /2 = • • • = /m and J^i^i = 1 (Zeeman, 1980; 
Hofbauer and Sigmund, 1998). For the extended replicator djTiamics, the interior equilibrium is 
not unique even when fi is linear in the density of each strategy Xi + yi, X2 + y2, ■ ■ ■, and Xm+Vm- 
Indeed, any configuration that satisfies /i = /2 = • • • = fm, Hi^i = 1? and J^iVi = ^ leads to 
dxi/dt = dyi/dt = in Eqs. and for G given in Eq. ([3]). Because there are 2m unknown 
variables and m + 1 equations, the equilibrium is underdetermined. Only the proportion of 
each strategy with different adaptation rates pooled, namely, Xi + yi, is determined by the 
equilibrium condition. 

To avoid this ambiguity, we introduce a realistic assumption for G. Because of bounded 
cognitive ability and stochastic environments, players may mimic strategies of unsuccessful 
others with small probabilities. This factor can be implemented by allowing G{x) > for x < 
(Blume, 1993; Nowak et al., 2004; Ohtsuki et al., 2006; Traulsen et al., 2006). More specifically, 
we assume that G is any differentiable function that satisfies G'{x) > 0, lim^^^-oo G'(x) = 0, 
and Eq. (jl]). This assumption implies that limx^ooG{x)/x = lim2,„>oo(a^ + G{—x))/x = 1 and 
G'{0) = 1/2. 

Then, it is straightforward to see that the interior equilibrium in the case of the linear 



8 



payoff is uniquely given by /i = = . . . = Z™, Ei "^i = 1, Ei Vi = 1, and Xi = yi {1 < i < m). 
The uniqueness can be shown as follows. If /i = /2 = • • • = /m does not hold at an interior 
equilibrium, there exists an such that fi^ > J2j fj^j and > J2j fjVj at this point. For this 
io, summation of Eqs. ([5]) and IQ yields 

° - + - ^« - S + «... (/... - E f,«) > 0. (7) 

which is a contradiction. Therefore, /i = /2 = • • • = /m holds. Then the first terms in the 
right-hand sides of Eqs. and vanish, and we obtain Xi = Ui {1 <i <m). 

3 Analysis of the RSP games 

We consider the symmetric RSP game whose fitness fi is specified via the following payoff 
matrix: 

/ -a2 63 
A=\ hi -a; 
\ -ai 62 

(Zeeman, 1980; Hofbauer and Sigmund, 1998). Each player can take one of the three strategies 
(m = 3), and the row player and the column player correspond to the focal player and the 
opponent, respectively. 

The unique interior equilibrium of the extended replicator dynamics is given by 





1^ 






X2 




v 


^3 ) 





1 



^ 0203 + a-sh + hh 

aia^ + aibs + 6163 | , (9) 
\ aia2 + 02^1 + bih 

where S is the normalization constant. In the standard replicator dynamics, this equilibrium 
is asymptotically stable when 010203 < &1&2&3 (Zeeman, 1980; Hofbauer and Sigmund, 1998). 
If 010203 > 61&2&3, the equilibrium is unstable, and in a finite population, the population 
density oscillates with ever increasing amplitudes until the population consists entirely of a 
single strategy. The relation 01O2O3 = &162&3 stipulates a borderline situation in which the 
interior equilibrium is neutrally stable. 
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By a one-to-one mapping of the population density vector, the payoff matrix A can be 
transformed to the following form (Zeeman, 1980; Hofbauer and Sigmund, 1998): 

(0 -a2 ai + e \ 
a2 + e -as (10) 
-ai as + e / 

Now, the asymptotic stability of the interior equilibrium is equivalent to e > 0. 

To show that heterogeneous adaptation rates stabilize the interior equilibrium, we even 
simplify Eq. ( fTOj) by setting CLl — (l2 — as — 1 (Zeeman, 1980): 




A" = + e -1 I (11) 
The interior equilibrium is now given by 

(12) 





xl \ 










v 


^3 J 






which is asymptotically stable if and only if e > 0. When e = 0, the equilibrium is neutrally 
stable, with A" defining a zerosum game. The standard replicator dynamics for e = becomes 
the most famous RSP dynamics: 



dxi 

~dt 

dx2 

~dt 

dxs 

~dt 

For A" given in Eq. ffTTj) . we obtain 



Xi{X'i-X2), (13) 
X2{xi-X:i), (14) 
X3{x2-Xi). (15) 



/i = (l + e)(xs + i/s) - (a;2 + y2), (16) 
/2 = (l + e)(xi + i/i)-(xs + z/s), (17) 
h = (l + e)(x2 + 2/2)-(xi+i/i). (18) 
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We linearize the extended replicator dynamics around the interior equilibrium by setting Xj 
X* + Axi, Hi = y* + Ayi {1 < i < 3), where Axi and Ayi are small. Using Axi + Ax2 + Ax^ 
Ayi + Ay2 + Ay^ = and G"(0) = 1/2, we obtain 

1 dAxi 



Px dt 
1 



Axi [(1 + e) (Ax3 + Ays) - (Axa + Ays) 



e (Axi + Ax2 + Axs) - - {Ay^ + Ay^ + Ay^) 



+ 



^ + Ayi) [G (0) + G' (0) (A/i - A/,)] + Ax,) 
- + ^^i) E (0) + (0) (A/, - A/O] + Ay,) 

2(1 + 6) 



+ 



3 

2(1 + 6) 



G(0) 
+ ^(0) 



2(2 + e)^ 



(19) 



where A/j is perturbation of the steady-state fi that originates from Axi, Ax2, and Axs. By 
doing similar calculations for Ax2, Ayi, and Ay2, we obtain the linear dynamics around the 
equilibrium given by Eq. ( |T2l) as follows: 



d_ 

dt 



/Axi\ /-(l + e)5,-a -(2 + e)E, -{l + e)B, + C, -(2 + e)5,. \ / Axi \ 



AX2 

Ayi 
V Ay2 / 



V 



(2 + e)B., 
(2 + e)By 



y I 



Bx — Cx (2 + t)Bx Bj- + C3, 

-(2 + e)5, -(l + e)S,-C, -(2 + e)5, 

-By + Cj/ (2 + e)By By — Cy j 



AX2 

Ayi 
V Ay2 / 

(20) 



where 5^ = 2/3^./3, = 2/3^^/3, = /3^G(0), and Cy = /3yG{0). Noting that B^Cy = ByC^ 
we obtain the following eigenvalue equation: 



A^+[2 {Cx + Cy) + e{Bx + By)] 



+ 



3 + 3e + e')[B'^ + B'] + {G, + GyY + 6 + 6e + B^By 



+ e {B^Gx + -Bi/C'j/ + 3BxGy + SByG^)] 



+ 



6 (2 + e)' B,By (a + Gy) + 4e (5, + By) G,Gy X + 12(2 + efB,ByG,Gy = 0. (21) 
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The interior equilibrium is stable if all the solutions of Eq. fl?T|) have negative real parts. 
The Routh-Hurwitz criteria dictate that all the eigenvalues of Eq. (pij) are negative if and 
only if the four principal minors of a matrix calculated from the coefficients of Eq. (|2T|) are all 
positive. To the first order of e, the principal minors are calculated as: 

\Hi\ = 2(C, + Cy) + e{B, + By), (22) 

\H2\ = Q{l + e){B,-Byf{C, + Cy) + 2{C, + Cy)' + 3e{B, + Byf 

+ eiB,, + By){3Cl + lOa^Cy + 3C^) (23) 

\Hs\ = 144(1 + 2e)iB, - ByfB^By{C^ + Cyf + 48(1 + e)B,By{C, + Cyf{C^ - Cyf 

+ 72e(5, + ByfB,By{C, + Cy) + 24e(5, + By)B,By{C, + Cy){2C^, + 3C,Cy + 2Cyf 
+ 24e(fi, + By) {B^ - Byf{C, + Cy)C.,Cy + 8e(5,, + fij,) (C, + CyfC^Cy, (24) 

li/^l = 48(l + e)E,E^C,C,|if3| (25) 

These relations hold whenever e > 0, that is, when the interior equilibrium is stable even 
without heterogeneous adaptation rates. When e < 0, the interior equilibrium is unstable with 
homogeneous adaptation rates. In accordance, \H^\ < holds when B^ = By > 0, Cx = Cy > 0, 
and e < 0. To make \Hs\ positive for e < 0, the heterogeneity quantified by {B^ — ByY and 
{Cx — CyY, must be large enough to compensate the negative contribution of the last four e 
terms of Eq. ([241) to li^s]. 

Although Eqs. (!22l)-(l25l) are valid for infinitesimally small |e|, they are exact when e = 0. In 
this case, the standard replicator dynamics have a neutrally stable interior equilibrium (Zeeman, 
1980; Hofbauer and Sigmund, 1998). Actually, {H^l = holds when B^ = By > 0, = Cy > 0, 
and e = 0. Equations imply that the asymptotic stability is equivalent to B^. ^ By 

and Cj. 7^ Cy, or equivalently, fix 7^ i^y, in addition to > and > 0. These analytical 
results indicate that heterogeneous adaptation rates stabilize the interior equilibrium. 
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4 Numerical results 



In this section, we perform individual-based numerical simulations in which players are involved 
in evolutionary games with cyclic competition. As a rule of thumb, when the population size 
is of the order of 100 or smaller, the finite-size effect tends to make amplitudes of oscillatory 
population densities explode. We show that, even in this case, introduction of heterogeneous 
adaptation rates prohibits the monopoly by a single strategy, which would follow unrestricted 
growths of oscillation amplitudes. As a result, different competing strategies can coexist. In 
the following numerical simulations, we assume n = 200 players. 

4.1 RSP games 

Consider that players are involved in the RSP game whose payoff matrix is given by Eq. ( ITTl) . 
Each player takes one of the three strategies. Initially, players select each strategy randomly 
and independently with probability 1/3. Any pair of players is engaged in the game with 
probability 0.5 so that each player is matched with 0.5{n — 1) = 99.5 other players on average. 

In accordance with the replicator dynamics, players with larger accumulated payoffs can 
disseminate their strategies more successfully. We pick 40 pairs of players randomly from the 
population and denote one such pair by i and j. Only player i is assumed to be subject to 
strategy update in this pairing. Player i copies player j's strategy with probability — fi), 
where /3j is the adaptation rate of player i. If fi > fj, no imitation occurs. This update rule is 
equivalent to the replicator dynamics with the piecewise linear G given in Eq. Although 
we required smooth G for formulating extended replicator dynamics, here we use the piecewise 
linear G for simplicity. The average adaptation rate is set equal to f3 = 0.5/n = 0.0025, 
and the adaptation rate of player i, or is distributed according to the uniform density on 
/? (1 — Ajs) ,/?(l + Ajj) , independently for different players. As Ap increases, the population 
becomes heterogeneous. 

For e = 0, the interior equilibrium is neutrally, but not asymptotically, stable in the standard 
replicator dynamics. The results of individual-based simulations are shown in Fig. [1] for three 
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values of heterogeneity A^. In Fig. [T], only the proportions of players with strategy 1 (first- 
row players in Eq. fllip ) are shown for visibility. In reality, three subpopulations, respectively, 
corresponding to strategies 1,2, and 3 alternately prosper due to cyclic competition. 

When = (Fig. [11(A)), all the players share the same adaptation rate (/3j = P) as in the 
standard replicator dynamics. Owing to the finite-size effect, the oscillation grows in amplitude 
until one strategy dominates the population after some oscillation cycles. The proportion of 
strategy 1 presented in the figure reaches or 1 depending on initial conditions and stochasticity 
in strategy updating. As the adaptation rate becomes heterogeneous, explosion of the oscillation 
amplitude tends to be suppressed. When A/? = 0.4 (Fig. [H^B)), oscillations persist for much 
longer time than when Ajj = (Fig. [D^A)). With more heterogeneity, the oscillation persists 
even longer and the oscillation amplitude becomes smaller, as shown in Fig. [1](C) for = 0.8. 

Precisely speaking, the theory predicts damping oscillations for heterogeneous adaptation 
rates, but not reverberating ripples apparent in Fig. WlB) and (C). There are many factors that 
could contribute to this discrepancy: the nonsmooth G, more than two values of adaptation 
rates, stochasticity introduced by random encountering and updating of randomly selected 
players, and small n. Because the aim in the numerical simulations is to show suppression of 
oscillations by heterogeneity, we do not explore this subtle discrepancy. 

For e = —0.1, the interior equilibrium is unstable in the standard replicator dynamics. In a 
homogeneous population, two of the three strategies are eradicated in an early stage (Fig. [21(A); 
A^ = 0). Even with some heterogeneity in the adaptation rate, the oscillation ends up with 
explosion (Fig. [2]^B); A^ = 0.4). At this level of heterogeneity, coexistence generally lasts 
longer than the homogeneous case, whereas its duration depends pretty much on numerical 
runs (results not shown). With stronger heterogeneity (Fig. [2]^C); A^ = 0.8), three strategies 
coexist stably with a high probability. 
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4.2 Public goods game with voluntary participation 

As another numerical example, we examine the public goods game, which is a type of the 
multiperson Prisoner's Dilemma. We assume that N {< n) randomly selected players form 
a group. In the standard public goods game, each of N players has an option to donate a 
unit cost or to refrain from donation. The donated amount is multiplied by r and divided 
equally by the N players. When r > 1, donation is a prosocial action because it increases 
the aggregated benefit of the group by proportion r — 1. However, a player always earns more 
by not donating, which is a social dilemma. In an evolutionary framework, the public goods 
game is repeated with different random groups in one generation. Then, the players with 
higher accumulated payoffs have more chances to disseminate their strategies. Without further 
assumptions, altruistic donation is completely overridden by defection. 

Incorporation of another strategy called loner opens the way to survival of cooperators 
(Hauert et al., 2002a; Haucrt ct al., 2002b). A loner does not participate in the game, and it 
gains the side payoff a > 0. The key assumption is that it is better not to play than to be 
in a group of defectors. If there are N — S loners in a group, the other S players play the 
public goods game. If there are ric cooperators among S players given S > 2, the payoffs for 
a cooperator, defector, and loner, are equal to /c = — 1 + rric/S, fd — rric/S, and // = a, 
respectively. If S' = 1, the single player constituting the group behaves like a loner because 
there is nobody to play the game with. 

With (7 < r — 1, it is better to stay in a group of cooperators than not to participate. There- 
fore, defectors dominate cooperators, as before, and loners dominate defectors, and cooperators 
dominate loners, which implies the RSP relation. According to the meanfield analysis with 
the standard replicator equations, the coexistence equilibrium is neutrally stable (Hauert et 
al., 2002a; Hauert et al., 2002b). The stable oscillation of the population density numerically 
shown for n — 5000 (Hauert et al., 2002a) may collapse in small populations. Oscillations can 
be stabihzed by placing players on the regular lattice (Szabo and Hauert 2002) or small-world 
networks with sufficient spatial structure (Szabo and Vukov 2004; Wu et al., 2005) (see (Szabo 
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and Fath, 2007) for more references). Here we are concerned to an alternative: the heterogene- 
ity mechanism. 

We perform numerical simulations with n = 200, = 5, r = 3, and a = 1. Initially, 
each player selects each strategy with probability 1/3. The generation payoff of player i is 
determined as the summation of the payoff after 2000 rounds of group formation. As in the 
numerical simulations of the RSP game (Sec. 14.11) . 40 players are subject to strategy update at 
the end of each generation. The average adaptation rate is set equal to (3 = 0.2N/n = 0.005. 

The results are shown in Fig. [3] for different degrees of heterogeneity in the adaptation 
rate. In each panel, cooperators (thin solid lines) are devoured by defectors (thin dashed lines), 
defectors are devoured by loners (thick solid lines), and loners are devoured by cooperators. 
This completes one RSP cycle. When the adaptation rate is homogeneous as in the standard 
replicator dynamics, the oscillation expands due to the finite-size effect. Two of the three 
strategies finally disappear (Fig. [3](A); Ap = 0). Unless r is so large that loners are not needed 
for the survival of cooperators, loners are relatively stronger than cooperators and defectors 
without violating the RSP relationship (Hauert et al., 2002a; Hauert et al., 2002b). Accordingly, 
the strategy that dominates finite populations are usually loners, which contrasts to the case of 
the RSP game. When the adaptation rate is heterogeneous (Fig. [HJ^B); = 0.4), oscillations 
typically last for longer time. With a stronger degree of heterogeneity (Fig. E^C); = 0.8), 
oscillations are more stable. In this way, heterogeneity promotes survival of cooperators in 
small populations of players in the Prisoner's Dilemma. 

5 Discussion 

5.1 Summary of the results 

We have analyzed evolutionary game dynamics of cyclically competing strategies (or pheno- 
types) with heterogeneous adaptation rates. We have shown analytically and numerically that 
such heterogeneity suppresses oscillations of the density of each strategy and dominance of a 
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particular strategy, which often occur in standard evolutionary dynamics with cyclic compe- 
tition. In a game such that a specific strategy is stable with a large attractive basin, players 
with large adaptation rates shift to the lucrative strategy quickly. Then they can transiently 
exploit conservative players with small adaptation rates. A larger adaptation rate is better in 
this situation. However, in games with cyclic competition, an obvious hierarchy of strategies is 
absent. Then conservative players contribute to diversification of a population without really 
sacrificing their own benefits. 

The assumption that different players have different adaptation rates seems realistic. There- 
fore our results imply that cyclic competition does not necessarily lead to apparent oscillations 
of population densities or dominance of particular strategies, which many analytical models 
predict. The other way round, an ordinary situation where three or more types of strategists 
stably coexist does not exclude operation of cyclic competition that is not weak. 

Generally speaking, evolutionary dynamics in finite populations deviate from theoretical 
predictions for infinite populations (Nowak et al., 2004). In homogeneous populations, RSP 
dynamics that show stable oscillations in infinite populations usually show unstable oscillations 
in finite populations, so that all but one phenotypes will disappear (Reichenbach et al., 2006). 
In the context of evolutionary games, altruistic strategies that are relatively weak in the RSP 
relation are expelled in finite populations, even if they can survive in infinite populations. The 
public goods game with voluntary participation investigated in Sec. 14.21 is such an example. 
Heterogeneity saves rare strategies particularly in small populations (up to the order of 100 
players) . 

5.2 Oscillations induced by mutation 

In addition to heterogeneous adaptation rates, there are mechanisms that sustain rare strategies 
in evolutionary dynamics with cyclic competition. One is spatial structure of contact networks, 
as explained in Sec. [T] (Hassell et al., 1991; Tainaka, 1993; Durrett and Levin, 1998; Frean and 
Abraham, 2001; Szabo and Hauert 2002; Szabo et al., 2004; Szabo and Vukov 2004; Wu et al.. 
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2005; Ying et al., 2007). Another important mechanism is mutation. No strategy dies out in 
the presence of mutation because mutation decreases the proportions of major strategies and 
increases those of rare and absent strategies. Actually, mutation establishes oscillations and is 
a key to altruism in the iterated Prisoner's Dilemma with cyclically competing three strategies, 
that is, ALLC, ALLD, and TFT (Imhof et al., 2005). 

We have neglected mutation in this work. Mutation- induced oscillations, which imply coex- 
istence of multiple strategies, and oscillations by heterogeneous adaptation rates are consistent. 
When mutation creates an oscillation in otherwise nonoscillatory evolutionary dynamics, the 
amplitude of an oscillation is fairly large because a mutation rate is typically small. A large 
oscillation amplitude implies that, in most of the time, the proportion of at least one strategy 
is very small (Imhof et al., 2005). With this case included, heterogeneity lessens the oscillation 
amplitude so that the proportion of each strategy does not fluctuate so much in time. 

5.3 Heavily skewed cyclic competition makes coexistence difficult 

To test our theory, we examined two numerical models in Sec. HI One is the standard RSP game 
that directly corresponds to the analytical model in Sec. [31 The other is the public goods game 
with voluntary participation (Hauert et al., 2002a; Hauert et al., 2002b). There are other evo- 
lutionary games in which oscillations derived from cyclic competition are observed. Examples 
include the Prisoner's Dilemma with a mixture of different memory-one strategies (Nowak and 
Sigmund, 1989a; Nowak and Sigmund, 1989b; Nowak and Sigmund, 1993), a seminal model of 
indirect reciprocity (Nowak and Sigmund, 1998a; Nowak and Sigmund, 1998b), and the iter- 
ated Prisoner's Dilemma with misimplementation of the action (Brandt and Sigmund, 2006). 
In these models, three or more strategies including altruistic strategies alternately prosper in 
infinite well-mixed populations. By additional numerical simulations, these dynamics were ex- 
amined with 200 < n < 1000 and heterogeneous adaptation rates. However, it was impossible 
to stabilize the oscillatory dynamics with the ranges of parameter values probed (results not 
shown) . 
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We conjecture that this failure is caused by the extreme asymmetry inherent in these RSP 
dynamics. To illustrate, suppose a population with homogeneous adaptation rates. By sym- 
metric RSP dynamics, we mean those defined by the payoff matrix given in Eq. f|TT]) . The 
corresponding dynamics of the population densities are schematically depicted in Fig. ID^A) so 
that the interior equilibrium is located in the center of the simplex. Each point in the sim- 
plex specifies a density profile of different strategies. A point close to the corner labeled 1, 
for example, corresponds to a population that contains relatively many players with strategy 
1. The symmetric or quasi symmetric RSP dynamics are often employed in the context of 
interacting particle systems as well as evolutionary games (Tainaka, 1993; Durrett and Levin, 
1998; Frean and Abraham, 2001; Szabo et al., 2004; Reichenbach et al., 2006; Ying et al., 
2007). An example of heavily asymmetric, or skewed, RSP dynamics is depicted in Fig. 111(B). 
To complete a cycle, the trajectory has to proceed through a narrow canal between the interior 
equilibrium and the heteroclinic path on which only strategies 2 and 3 exist (the bottom line 
of the triangle). Accordingly, the proportion of strategy 1 is close to zero for some significant 
time in each cycle. Such heavily skewed RSP dynamics are identified in literature (Nowak and 
Sigmund, 1989a; Nowak and Sigmund, 1989b) and may be widely found in evolutionary games. 

Because trajectories for a population of small to intermediate size accompany considerable 
fiuctuation, it likely hits the heteroclinic path so that the rare strategy (strategy 1 in Fig. |1](B)) 
perishes. If the location of the interior equilibrium and trajectories derived from an infinite- 
population theory are extremely skewed, oscillatory dynamics will not be realized in finite 
populations. Because the heterogeneous adaptation rate does not change the position of the 
equilibrium (see Sec.|3]and (Masuda and Konno, 2006)), heterogeneity does not widen the canal. 
Although mutation kicks trajectories that have fallen onto the heteroclinic path back to the 
interior of the simplex, the rare strategy is perpetually subject to extinction. The heterogeneity 
mechanism discovered here will work out for RSP dynamics that are not extremely skewed. 
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Figure 1: Evolutionary simulations of the zerosum RSP game in finite populations. We set 
n = 200, e = 0, and (3 = 0.0025. The dispersion of the adaptation rate is (A) absent (A^ = 0), 
(B) intermediate (A^ = 0.4), and (C) large (A^ = 0.8). For clarity, the proportion of the 
players of only one of the three strategies is shown in each panel. The horizontal lines indicate 
1/3. 
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Figure 2: Evolutionary simulations of the non-zerosum RSP game in finite populations. We 
set n = 200, e = —0.1, /? = 0.0025. The dispersion of the adaptation rate is equal to (A) 0, 
(B) 0.4, and (C) 0.8. 
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Figure 3: Evolutionary simulations of the public goods game with voluntary participation in 
finite populations. We set n = 200, AT = 5, r = 3, (t = 1, ^ = 0.005, and (A) = 0, (B) 
= 0.4, and (C) A^ = 0.8. The proportions of cooperators, defectors, and loners, are shown 
by thin solid lines, thin dashed lines, and thick solid lines, respectively. 
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Figure 4: Schematic of trajectories of (A) symmetric and (B) heavily skewed RSP dynamics. 
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